Augmented 퓁1 and Nuclear-Norm Models with a Globally Linearly Convergent Algorithm

This paper studies the long-existing idea of adding a nice smooth function to “smooth” a nondifferentiable objective function in the context of sparse optimization, in particular, the minimization of ‖x‖1 + 1 2α‖x‖ 2 2, where x is a vector, as well as the minimization of ‖X‖∗+ 1 2α‖X‖ 2 F , where X is a matrix and ‖X‖∗ and ‖X‖F are the nuclear and Frobenius norms of X, respectively. We show that they let sparse vectors and low-rank matrices be efficiently recovered. In particular, they enjoy exact and stable recovery guarantees similar to those known for the minimization of ‖x‖1 and ‖X‖∗ under the conditions on the sensing operator such as its null-space property, restricted isometry property, spherical section property, or “RIPless” property. To recover a (nearly) sparse vector x, minimizing ‖x‖1+ 1 2α‖x‖ 2 2 returns (nearly) the same solution as minimizing ‖x‖1 whenever α ≥ 10‖x‖∞. The same relation also holds between minimizing ‖X‖∗+ 1 2α‖X‖ 2 F and minimizing ‖X‖∗ for recovering a (nearly) low-rank matrix X if α ≥ 10‖X‖2. Furthermore, we show that the linearized Bregman algorithm, as well as its two fast variants, for minimizing ‖x‖1 + 1 2α‖x‖ 2 2 subject to Ax = b enjoys global linear convergence as long as a nonzero solution exists, and we give an explicit rate of convergence. The convergence property does not require a sparse solution or any properties on A. To our knowledge, this is the best known global convergence result for first-order sparse optimization algorithms.